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1.0  EXECUTIVE  SUMMARY 


1.1  BACKGROUND 

Detection  of  unexploded  ordnance  (UXO)  and  discrimination  between  UXO  and  metallic  clutter 
pose  challenges  to  cost-effective  remediation  of  UXO-contaminated  land  in  order  to  reclaim  it 
for  civilian  use.  The  need  to  develop  sensors  that  can  be  deployed  in  a  realistic  setting  and 
accomplish  these  goals  motivated  this  project. 

The  broadband  electromagnetic  sensor  improvement  and  demonstration  undertaken  in  this 
project  took  the  prototype  GEM-3  and  evolved  it  into  an  operational  sensor  with  increased 
bandwidth  and  dynamic  range,  and  enough  memory  and  processing  power  to  allow  efficient  data 
acquisition  while  decreasing  the  weight  for  ease  of  operation.  Specifically,  the  frequency  range 
was  doubled  from  24  kHz  to  48  kHz;  the  24-bit  analog-to-digital  converter  (ADC)  replaced  the 
16-bit  ADC,  providing  48  dB  increased  dynamic  range;  the  digital  signal  processor  (DSP)  was 
upgraded  to  be  fast  enough  to  allow  continuous  operation  at  more  than  10  frequencies 
simultaneously;  and  compact-flash  memory  was  added,  accommodating  internal  storage  of  a 
day’s  surveying.  In  addition  to  the  original  handheld  configuration  specified  at  the  outset,  a  cart- 
mounted,  large-coil  configuration  has  also  been  developed  for  surveying  large  open  areas.  A 
more  detailed  description  is  provided  in  Section  2.0,  Technology  Description. 

Other  new  capabilities  include  real-time  output  of  data  for  logging  into  a  portable  computer 
(either  laptop  or  palm)  as  an  option  to  internal  storage.  Seamless  integration  with  global 
positioning  system  (GPS)  provides  geo-referenced  data  in  Universal  Transverse  Mercator  (UTM) 
grid  coordinates.  Utility  software  for  both  the  laptop  and  the  palm  computers  provides  an 
advanced  user  interface  for  configuring  and  controlling  the  GEM  as  well  as  real-time  data 
display  and  processing.  Mechanical  construction  was  refined  as  well. 

The  demonstration  was  designed  to  test  the  technology  in  a  wide  range  of  operational 
requirements,  including  rugged  terrain  (moguls),  wooded  areas,  large  open  areas,  and 
miscellaneous  obstacles  and  challenges  (water,  fences,  roads,  etc.).  To  meet  these  requirements, 
multiple  configurations  of  the  technology — handheld,  manually  pushed  cart,  and  all-terrain 
vehicle  (ATV)  towed  sled — were  implemented  and  tested. 

1.2  OBJECTIVES  OF  THE  DEMONSTRATION 

Through  this  program,  the  GEM-3  has  evolved  from  prototype  into  a  fully  operational,  full-scale 
instrument,  and  the  objective  of  this  demonstration  was  to  verily  its  capability  to  perform  under 
realistic  conditions  of  buried  UXO  contamination.  The  goal  was  to  combine  multi-frequency 
inphase  and  quadrature  data  in  an  optimal  way  to  identify  local  anomalies  that  are  potentially 
UXO. 

The  technology  demonstration  scope  was  all-encompassing,  including  blind-grid  testing  of  three 
configurations — handheld  40cm  sensor,  pushcart  mounted  96cm  sensor,  and  ATV  towed 
96cm  sensor  challenge  scenarios — with  woods  and  moguls  using  the  handheld  sensor  and  large 
open  fields  using  the  large  coil  with  pushcart  and  towed  configurations.  The  moguls  area  was 
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covered  with  snow,  the  woods  and  parts  of  the  open  area  were  covered  with  several  inches  of 
water. 


1.3  REGULATORY  DRIVERS 

This  program  was  undertaken  in  response  to  the  Environmental  Security  Technology 
Certification  Program  (ESTCP)  Topic  1:  Unexplored  Ordnance  (UXO)  Detection, 

Discrimination,  and  Remediation. 

1.4  DEMONSTRATION  RESULTS 

The  new  features  of  the  GEM-3  were  proven  to  be  fully  functional — lightweight,  high  dynamic 
range  and  sensitivity,  ability  to  record  10  frequencies  continuously  and  simultaneously  from  90 
Hz  to  41  kHz,  high  data  storage  capacity,  tightly  integrated  differential  global  positioning  system 
(DGPS),  and  advanced  user  interface  software  for  the  handheld  computer  as  well  as  PC-based 
configuration  and  operational  software.  Handheld  as  well  as  cart-  or  sled-mounted  pushed  or 
towed  configurations  were  developed  and  tested  for  rough,  vegetated,  and  open  terrain  missions. 
Discrimination  software  that  can  run  real-time  on  the  handheld  computer,  or  in  a  batch  post 
processing  mode,  and  an  automatic  target-picking  algorithm  were  developed. 

The  results  of  the  demonstration  indicate  a  need  for  further  work.  Perfonnance  in  the  moguls 
and  woods  was  not  consistent  with  our  other  experiences  with  detection  capability  of  the  hand¬ 
held  GEM-3,  and  we  believe  a  problem  with  our  target  geo-referencing  may  have  occurred.  We 
detected  a  large  number  of  targets  in  these  areas  that  had  the  characteristics  of  metal  objects  but 
did  not  match  the  ground  truth  positioning  of  seeded  objects  (UXO  or  clutter),  indicating  a 
probable  problem  with  our  geo-referencing.  We  have  since  repeated  the  demonstration  in  the 
moguls  area  (April  2005)  with  some  small  improvements,  mainly  in  the  algorithms,  to  the 
system,  and  achieved  a  UXO  detection  of  nearly  70%.  Detection  performance  was  reasonable  in 
the  other  areas,  but  discrimination  capability  has  yet  to  be  realized.  (The  Aberdeen  Proving 
Ground  [APG]  site,  with  its  wide  range  of  ordnance  types,  is  particularly  challenging.)  Further 
system  improvements  that  may  help  accomplish  this  include  increased  range  capability  for 
detection  of  deeper  targets  and  improved  detection  channel  algorithms  and  discrimination 
algorithms.  Acquisition  methodologies  also  must  be  improved;  we  must  ensure  that  the  geo- 
referencing  is  correct,  and  improve  the  cart  and  towing  platforms  and  ATV  navigation. 

Detailed  performance  results  are  provided  in  Section  4.0,  Performance  Assessment. 

1.5  STAKEHOLDER/END-USER  ISSUES 

This  demonstration  addressed  decision-making  issues  concerning  end  users  associated  with  the 
applicability  of  the  GEM-3  technology  for  their  specific  UXO  detection  and  discrimination 
mission  needs.  Operational  performance  under  a  variety  of  conditions  was  assessed,  including 
production  capabilities,  field  usability,  and  logistical  requirements.  Both  handheld  (with  small 
sensor  disk)  and  cart-  or  sled-mounted  (large  sensor  disk)  configurations  were  evaluated, 
providing  end-user  assessment  of  the  performance  and  operational  usability  of  each. 
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2.0  TECHNOLOGY  DESCRIPTION 


2.1  TECHNOLOGY  DEVELOPMENT  AND  APPLICATION 

2.1.1  Hardware 

The  GEM-3  electromagnetic  induction  (EMI)  system  consists  of  three  basic  components:  the 
monostatic  coil  sensor,  the  electronics  console,  and  the  user  interface  (control  and  display) 
module  (Figure  1).  The  functional  architecture  is  described  by  Won  et  al,  1997,  and  the  design 
details  of  the  improved  system  are  given  in  the  Final  Report  for  this  project. 


Figure  1.  Schematic  of  the  GEM-3  (top),  Conceptual  Representation  of  the  GEM-3  Coils  (bottom 
left),  and  Console  Block  Diagram  Showing  Functional  Modules  (bottom  right).  (The  GEM-3  coils 
create  a  central  magnetic  cavity  region  using  two  concentric  circular  loops  that  are  electrically 

connected  in  an  opposing  polarity.) 

Although  a  new  custom  user  interface  was  designed  and  built  for  the  prototype,  the  full-scale 
system  utilizes  a  palmtop  type  computer  such  as  the  iPAQ™,  with  custom  software  for 
configuring  the  GEM,  and  for  real-time  control  and  display.  The  new  hardware  will  also  allow 
the  use  of  a  12V  standard  off-the-shelf  NI-mH  battery.  The  improved  electronics  is  more 
compact  and  lighter  than  earlier  designs,  provides  increased  dynamic  range  and  bandwidth  via 
24-bit  ADC  with  96  kHz  sampling,  incorporates  a  preamp  built  into  the  receiver  (Rx)  coil  to 
reduce  cable-noise  pickup,  and  uses  more  efficient  and  faster  switching  transmitter  (Tx)  field 
effect  transistors  (FET)  for  less  heat  and  greater  waveform  fidelity.  Photographs  of  the 
electronics  are  shown  in  Figure  2. 
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Figure  2.  Photographs  of  the  Electronics  Console  of  the  Enhanced  GEM-3  (left)  and  the  DSP 

(right).  (With  cover  removed  from  console,  the  memory  card  and  support  electronics  on  the  interior 

horizontal  board  are  viewable.) 

Through  this  program,  the  GEM-3  has  evolved  from  prototype  into  a  fully  operational  (full- 
scale)  instrument,  and  the  objective  of  this  demonstration  was  to  verify  its  capability  to  perform 
under  realistic  conditions  of  buried  UXO  contamination.  The  goal  was  to  combine  multi¬ 
frequency  inphase  and  quadrature  data  in  an  optimal  way  to  identify  local  anomalies  that  are 
potentially  UXO. 

The  GEM-3  may  be  used  in  two  versions — handheld  with  either  40cm  or  64cm  diameter  sensor 
(40cm  for  this  demonstration)  or  cart-  or  sled-mounted  with  96cm  sensor,  as  shown  in  Figure  3. 
The  large-disk  configuration  can  be  mounted  on  a  PVC  sled  and  towed  with  an  ATV,  or 
mounted  on  wheels  and  hand-pushed  when  greater  maneuverability  is  required. 


Figure  3.  The  GEM-3  Can  Be  Handheld,  Cart-  or  Sled-Mounted,  and  Pushed  or  Towed. 

The  GEM-3  electronics  has  integrated  DGPS  navigation  capabilities,  although,  for  the  hand-held 
scenarios  of  this  demonstration,  we  flagged  anomalies  and  located  them  with  a  stand-alone 
DGPS  afterwards. 

The  gain  in  signal-to-noise  ratio  (s/n)  from  the  increase  in  maximum  Tx  current  depends  on 
several  factors.  Although  the  maximum  current  sustainable  by  the  electronics  is  doubled,  the 
actual  realized  increase  depends  on  the  set  of  frequencies  chosen  -  high  frequencies  are 
inductively  loaded  and  demand  a  greater  share  of  the  available  voltage  (12v)  and,  using  more 


4 


frequencies,  shares  the  voltage  and  decreases  the  peak  current.  With  a  10  frequency  hybrid 
waveform  spread  roughly  logarithmically  over  the  sensor  48  kHz  band  (typical),  the  current  will 
not  reach  12  A,  but  if  the  selection  is  restricted  to  a  few  frequencies  at  the  low  end,  the  maximum 
current  will  be  attained  and  the  benefit  will  be  more  clear  cut. 

The  GEM-3  transmitter  current  is  produced  by  computer  controlled  rapid  switching  of  the  full 
battery  voltage  into  the  Tx  coils,  with  a  current  response  characterized  by  the  natural  self¬ 
inductance  and  resistance  of  the  coil.  The  pulse-train  pattern  is  customized  to  deliver  current  at 
the  user-selected  frequencies.  The  higher  Tx  waveform  speed  offers  a  more  power  efficient  Tx 
system  because  a  lower  speed  (also  implies  longer  duration  pulses)  results  in  more  current 
overshoots  that  must  be  countered  with  current  pulses  of  opposite  polarity,  and  thus  more  power 
expended  at  (nonoperational)  switching  frequency.  Also,  a  slower  waveform  pulse  train  with 
longer  pulses  results  in  higher  primary  field  spikes,  which  consumes  much  of  the  receiver 
dynamic  range  with  residual  (bucking  error)  primary  field  induced  voltage. 

The  increase  in  sampling  rate  to  30  Hz  (with  an  option  to  average  internally,  reducing  the  output 
rate  but  narrowing  the  noise  bandwidth  around  each  operational  frequency)  provides  a  clear-cut 
increase  in  spatial  resolution  during  survey  mode  operation  and  greater  stacking  rate  for  s/n 
improvement  during  static  logging  over  a  target. 

The  24-bit  ADC  provided  much  needed  dynamic  range  without  sacrificing  sensitivity.  In  our 
early  systems  using  a  16-bit  ADC,  we  had  experienced  problems  with  sensor  saturation  in 
magnetic  terrain,  notably  Kaho’olawe,  Hawaii,  as  well  as  the  road  lanes  at  the  landmine  test  site 
at  Aberdeen  Proving  Grounds.  In  an  early  test  at  Kaho’olawe,  it  was  necessary  to  reduce  the 
electronics  gain,  thereby  reducing  sensitivity,  in  order  to  operate.  At  the  APG  mine  lanes,  we 
abandoned  testing  on  the  road  lanes  and  could  operate  only  over  the  off-road  (grass)  lanes. 
Large  shallow  metal  targets  can  also  saturate  the  ADC,  and  the  16-bit  version  had  a  much  lower 
threshold  before  this  limitation  was  reached.  The  doubling  of  the  ADC  speed  to  96  kHz  doubled 
the  operational  bandwidth  to  48  kHz  (we  are  now  moving  to  182  kHz  sampling  for  96  kHz 
bandwidth).  Many  targets  have  not  reached  the  inductive  limit  at  24  kHz;  i.e.,  the  spectral 
character  is  not  completely  defined  below  24  kHz,  and  so  the  added  bandwidth  should  enhance 
spectral-based  discrimination. 

The  advanced  display  has  been  replaced  by  integration  with  a  palm-held  computer  (e.g., 
iPAQ™). 

The  data  logging  function  allows  tight  integration  of  auxiliary  data  with  the  GEM  data;  the 
primary  application  thus  far  has  been  DGPS,  but  it  could  include  magnetometer  data  or  other 
sensors  that  utilize  an  RS-232  output. 

The  maximum  number  of  frequencies  goes  hand-in-hand  with  the  increased  bandwidth;  in  order 
to  take  advantage  of  the  bandwidth  without  sacrificing  spectral  resolution,  more  frequencies  are 
needed.  Also,  increasing  spectral  resolution  should  enhance  target  discrimination  based  on 
spectral  character,  particularly  when  several  different  types  of  targets  are  of  interest. 
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Integration  of  real-time  detection  (audio  enunciated)  and  discrimination  provides  an  operational 
prototype  of  an  advanced  system  for  locating  targets  and  prioritizing  a  dig  decision  on  the  spot. 
We  have  implemented  a  simple  single-point  spectral  matching  based  algorithm  (best- fitting 
linear  combination  of  axial  and  transverse  target  orientation  frequency  responses).  More 
advanced  schemes  using  multiple  spatial  samples  could  be  added. 

2.1.2  Software 

The  objective  of  the  demonstration  was  to  evaluate  both  the  GEM-3  hardware  as  well  as  the  data 
processing,  including  detection  and  discrimination  algorithms. 

2. 1.2.1  Response  Stage  Algorithms 

These  target-detection  algorithms  provide  a  means  to  combine  multifrequency  data  into  a  single 
output  measurement  that  is  used  as  the  response  stage  of  the  demonstration  submittals  (dig  lists). 
They  are  designed  to  suppress  geologic  noise,  particularly  soil  susceptibility,  and  to  emphasize 
metallic  objects,  and  increase  system  s/n  by  combining  many  independent  signals.  They  also 
drive  sound  and  graphical  display  signals  to  the  operator  for  real-time  detection  (hand-held 
configuration)  of  EMI  anomalies  that  may  indicate  buried  UXO,  or  anomaly-picking  criteria  in 
the  automated  target-picking  software  (towed  survey  configuration)  used  in  survey  data 
postprocessing. 

Soil  magnetic  susceptibility  is  the  predominate  source  of  geologic  noise  (i.e.,  EMI  anomalies 
caused  by  soil),  which  can  result  from  soil  variability  or  from  sensor  height  and  orientation 
changes.  The  response  stage  is  designed  to  suppress  geologic  noise.  In  general,  the  multi- 
frequency  EMI  response  of  magnetic  soil  is  characterized  by  zero  quadrature  and  frequency- 
independent  inphase  variations  (i.e.,  flat,  zero  phase  spectra).  A  weak  quadrature  response 
increasing  with  frequency  results  from  soil  conductivity.  Metal  objects  produce  relatively  strong 
quadrature  responses,  usually  extending  to  low  frequencies,  and  frequency-dependent  inphase 
responses  (Figure  4).  The  response  stage  algorithms  are  designed  to  ignore  variations  in  the 
absolute  level  of  the  inphase  (frequency-independent-level  shifts). 

Spectra  from  geologic  anomalies  have  a  very  flat  (frequency-independent)  inphase  character 
from  soil  susceptibility  and  a  weak  quadrature  response,  increasing  monotonically  with 
frequency  from  soil  conductivity.  Our  response  stage  algorithms  more  easily  mitigate  the 
former,  while  the  latter  is  mitigated  simply  by  its  relative  weakness. 
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Figure  4.  Example  of  Metallic  Target  Spectra  for  Sensor  Axis.  (The  target  cylindrical  symmetry >  axis 
[blue  is  longitudinal  and  red  is  transverse]  shows  the  strong  quadrature  and  frequency  dependent 
inphase  character.  The  green  shows  characteristic  idealized  geologic  response.) 

We  have  available  the  following  detection  (response  stage)  algorithms: 

•  Quadrature  Sum:  Sum  of  multifrequency  quadrature  responses,  which  are 
insensitive  to  geologic  magnetic  susceptibility,  weakly  sensitive  to  geologic 
conductivity 

•  Inphase  Difference:  Difference  between  high  and  low  frequency  inphase 
desensitizes  to  geologic  magnetic  susceptibility,  can  choose  frequencies  with  less 
noise  as  long  as  they  are  not  too  close 

•  Spread  Functions  (Quadrature  or  Inphase):  Sum  of  absolute  differences 
among  all  chosen  frequencies,  best  at  suppressing  geologic  noise,  but  random 
(high  spatial  frequency)  noise  increases 

•  Total  Apparent  Conductivity  (I-Q  and  Q-Q):  Nonlinear  transformations  tend  to 
bring  out  weaker  anomalies  (small  or  deep  UXO),  weighted  average  over  all 
frequencies,  l/log(f)  weighting,  which  favors  metal  over  geology. 

These  response  stage  measurements  do  not  provide  a  means  to  identify  or  classify  the  target 
unless  spatial  data  are  used  to  extract  shape  features  such  as  aspect  ratio.  Our  discrimination 
schemes,  described  below,  utilize  target  spectral  character  rather  than  multispatial  samples. 

In  all  these  algorithms,  previewing  the  individual  spectral  components  or  comparing  results 
(i.e.,  resultant  map  clarity,  which  is  somewhat  subjective),  allows  exclusion  of  problem 
frequencies.  For  the  survey  mode  data  (cart  or  towed),  maps  from  various  choices  can  be 
compared  since  the  algorithms  are  performed  in  postprocessing.  In  practice,  frequencies  below 
270  Hz  are  often  degraded  by  platform  motion  for  a  cart  mounted  system,  and  the  highest 
frequency,  near  the  system  band  limit,  is  often  noisier  than  the  others  and  more  sensitive  to 
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conductive  soil.  The  90  Hz  and  150  Hz  and  47,970  Hz  were  excluded  in  the  detection  channel 
(Qspread)  at  Kaho’olawe  and  the  lowest  (90  Hz)  and  highest  (41,010  Hz)  at  APG  for  the  total 
apparent  conductivity.  In  handheld  operation,  the  benign  platform  dynamics  allowed  use  of  all 
frequencies  at  APG. 

The  total  apparent  Q-Q  conductivity  is  considered  the  overall  optimum  response  stage  and  was 
used  for  the  open  field  at  APG,  in  which  the  survey  data  were  postprocessed  in  batch  mode. 
Real-time  implementation  at  that  time  required  the  use  of  the  simpler  quadrature  sum  in  the 
handheld  configuration  in  which  the  operator  picked  targets  during  the  survey;  recently,  we  have 
combined  the  quadrature  sum  with  the  inphase  difference.  When  we  have  compared  results  from 
these  three  algorithms,  the  perfonnance  difference  is  usually  insignificant.  At  Kaho’olawe,  we 
used  the  quadrature  spread,  which  showed  somewhat  less  broad  background  variation  than  the 
quadrature  sum,  at  the  price  of  greater  short  spatial  wavelength  noise. 

The  response  stage  magnitude  correlates  to  target  metal  content  (size)  and  sensor-target  range 
(depth);  the  spatial  extent  of  the  anomaly  correlates  to  target  depth. 

2. 1.2.2  Discrimination  Stage  Algorithms 

The  algorithm  used  for  discrimination  is  a  simple  fit  with  arbitrary  weights  to  a  mix  of  the  two 
library  spectral  response  modes  (transverse  and  longitudinal)  (Norton  et  al,  2001b).  Sensor- 
target  geometry  is  not  modeled,  so  data  sample  positions  are  not  needed,  nor  are  target  position 
and  orientation.  The  weighting  factors  were  unconstrained  (we  have  more  recently  constrained 
them  to  be  positive)  and  incorporate  response  amplitude  scaling  associated  with  target  range;  the 
relative  values  are  determined  by  target  orientation.  The  best-fit  library  target  for  a  set  of  samples 
acquired  at  the  peak  of  the  response  stage  is  determined,  with  goodness  of  fit  as  a  confidence 
criteria  computed  from  the  spectral  fitting  error  to  generate  the  discrimination  stage  (DS).  The 
DS  does  not  correlate  with  sensor  output  amplitude  as  does  the  response  stage  because  amplitude 
depends  primarily  on  target  size  and  depth,  and  both  UXO  and  clutter  occur  over  a  broad  range 
of  sizes  and  depths.  Our  DS  is  based  on  characteristic  multi-frequency  spectra  correlated  to 
UXO  training  (library)  spectra — the  ’’shape”  of  the  spectra  (relative  response  between 
frequencies)  detennines  which  library  ordnance  best  matches  the  target  and  how  well  it  fits. 

In  addition  to  the  weighting  factors,  the  algorithm  allows  a  frequency-dependent  inphase  offset 
(vertical  shift  in  the  inphase  spectrum)  to  desensitize  the  fit  to  superimposed  geologic  noise  from 
magnetic  terrain.  An  option  to  add  a  contribution  to  the  misfit  from  the  offset  allows  some 
recovery  of  the  absolute  inphase  level  (which  depends  on  the  ferrous  response  of  the  target) 
target  information  if  the  soil  susceptibility  is  not  too  high.  The  misfit  is  normalized  by  the  sum 
of  the  target  amplitude  plus  a  constant  adjusted  so  that  weak  targets  (small  or  deep)  allow  for 
greater  relative  (%)  noise. 

The  DS  for  the  APG  data  was  computed  in  a  batch  mode  from  single  points  identified  with  the 
response  stage. 

At  APG,  the  DS  was  computed  from  the  matching  misfit,  where  we  mapped  zero  fitting  error  to 
a  maximum  value  of  10  and  infinite  fitting  error  to  zero  via  the  form: 
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DS  =  C/(0.1*C  +  Error),  with  C  chosen  to  obtain  the  desired  DS  distribution. 


The  value  for  C  ideally  maps  the  UXO/clutter  threshold  to  a  confidence  (discrimination  stage) 
value  of  5,  which  gives  the  misfit  error  a  value  of  0. 1  *C. 

Since  APG,  we  have  modified  this  expression  by  normalizing  and  squaring  the  error: 

DS  =  10/(1  +  (Error/C)A2),  with  C  chosen  to  obtain  the  desired  DS  distribution. 

This  shifts  the  value  of  C  to  be  equal  to  the  UXO/clutter  error  threshold  (DS=5  when  error=C), 
and  it  flattens  the  DS  distribution  so  that  fewer  targets  give  a  DS  near  5,  and  more  targets  give 
values  near  the  extremes  (0  and  10).  The  objective  is  to  have  a  better  separation  of  UXO  and 
clutter  and  ideally  to  give  a  bimodal  DS  distribution  for  the  two  classes. 

At  Kaho’olawe,  the  total  misfit  was  computed  from  a  weighted  sum  at  multiple  positions,  where 
the  weighting  was  inversely  proportional  to  the  distance  from  the  peak  response.  Data  points 
included  were  manually  selected  for  each  target  using  a  cursor  selection  function  built  into 
custom  contour  mapping  software.  At  the  time,  we  had  not  yet  developed  batch-mode  software 
and  were  evolving  our  analysis  methodology;  the  manual  target-by-target  processing  was 
extremely  laborious,  and  the  benefit  in  results  not  clear;  and  we  found  it  difficult  to  treat  data 
consistently  over  the  many  days  of  processing  required.  That  motivated  the  development  of  an 
algorithm  that  could  pick  target  anomalies  and  perform  the  inphase  difference  (ID)  algorithm  and 
output  a  dig  list  with  target  coordinates,  response  stage  and  DS  values,  and  target  ID  in  a  batch 
mode.  The  DS  for  the  APG  data  was  computed  using  this  software,  with  target  IDs  computed 
from  single  points  identified  with  the  response  stage.  Use  of  a  single  point  is  much  more 
straightforward  for  an  automated  algorithm,  and  we  have  not  found  any  benefit  from  processing 
multiple  samples  for  the  same  target  in  our  testing  to  date,  though  with  more  research  an  optimal 
method  of  combining  spatial  samples  without  explicit  modeling  may  provide  improved 
classification  perfonnance. 

Figure  5  shows  examples  of  a  good  fitting  and  a  poorly  fitting  target  spectra  to  library  UXO,  the 
former  likely  UXO  and  the  latter  likely  clutter;  note  that  the  amplitude  (and  correspondingly,  the 
response  stage)  is  greater  for  the  likely  clutter  item. 
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Figure  5.  Examples  of  Blind  Grid  Target  Spectral  Fits  to  Library  Training  Spectra.  (The  top 
graph  indicates  a  good  fit  with  a  high  [8]  discrimination  stage  [confidence] ,  and  the  bottom  graph 
indicates  likely  clutter  with  a  low  [2]  discrimination  stage.) 
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2.2  PREVIOUS  TESTING  OF  THE  TECHNOLOGY 


Initial  testing  of  GEM-3  technology  was  first  performed  at  the  Geophex  facility  in  Raleigh, 
North  Carolina.  Geophex  has  a  10m  x  10m  test  bed  in  which  21  metal  pipes  of  various  sizes, 
some  ferrous  (steel)  and  some  nonferrous  (3  aluminum,  2  copper),  have  been  buried  at  depths 
ranging  from  10  to  1 10cm. 

The  prototype  of  the  full-scale  GEM-3  technology  was  first  demonstrated  at  the  Kaho’olawe 
demonstration  in  October  2001  for  the  Naval  Facilities  Engineering  Command's  Pacific  Division 
(PACDIV).  The  hardware  tested  incorporated  all  of  the  full-scale  components  except  the 
integrated  iPAQ™  user  interface  that  has  replaced  the  prototype  control  and  display  unit.  Minor 
hardware  changes  and  DSP  firmware  changes  have  been  made,  and  changes  have  been  made  to 
the  system  configuration  and  user-interface  (WinGEM)  software.  The  Kaho’olawe  site  included 
several  30m  x  30m  calibration  grids  and  22  blind  grids,  also  30m  x  30m  each,  of  varying 
(moderate-high)  geologically  magnetic  terrain.  The  demonstration  phase  at  that  site  was 
performed  in  October  2001. 

We  used  the  cart-mounted  96cm  disk  exclusively,  with  integrated  DGPS  positioning  of  survey 
data  recorded  by  the  GEM-3  electronics. 

2.3  PROCESS  DESCRIPTION 

Mobilization  requires  transport  of  the  GEM  sensors  (and  for  large  open  areas,  the  ATV  and 
towed  sensor  platform  (pvc  sled)  and/or  wheeled  pushcart)  and  auxiliary  equipment,  including 
battery  chargers,  laptop  computer,  DGPS  with  base  station,  lane  markers  (cord,  traffic  cones, 
plastic  flags,  etc.). 

Design  criteria  include  determining  configuration  choices — handheld  for  areas  not  easily 
surveyed  with  a  cart  or  towed  system,  cart  for  small  open  areas  where  maneuverability  is 
required,  and  ATV  towed  system  for  large  open  areas.  Line  spacing,  typically  a  half  meter,  may 
be  increased  if  only  larger  ordnance  is  of  interest,  or  tightened  if  small  shallow  items  are 
important.  The  detection  channel  and  discrimination  algorithm  settings  may  depend  on  soil 
characteristics  (susceptibility  and  conductivity). 

Performance  metrics  are  based  on  probability  of  detection  (PD)  of  UXO,  false  alann  rate  (FAR), 
and  survey  area  production  rate. 

Personnel  training  requirements  are  typical  of  commercial  geophysical  equipment,  with 
familiarity  of  equipment  setup  (i.e.,  assembly  of  cart,  cable  connections,  battery  and  power 
needs)  and  operation  (iPAQ™  and  PC  software,  data  file  management). 

Health  and  safety  issues  pertain  to  the  UXO  sites,  but  no  special  hazards  are  associated  with  the 
GEM  hardware. 
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2.4  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 


The  chief  advantage  of  this  technology  is  the  potential  for  discrimination  between  ordnance  and 
clutter,  and/or  identification  of  the  ordnance.  The  broadband  multifrequency  data  provides 
maximum  EMI  information  that  can  be  used  to  identify  or  classify  metal  objects.  The  high 
sampling  rate  provides  dense  coverage  when  conducting  a  production  survey,  and  the  dynamic 
range  allows  detection  of  small  or  relatively  deep  targets  while  remaining  below  saturation  even 
when  large  targets  are  shallow,  or  when  operating  over  highly  magnetic  soil.  The  various 
configurations  and  coil  sizes  available  make  the  system  adaptive  to  differing  conditions  (hand¬ 
held  in  trees  or  over  rough  terrain,  large  cart-mounted  disk  providing  increased  footprint  and 
depth  of  exploration  for  flat  open  areas).  Major  improvements  under  this  project  include  the 
following: 

•  Higher  maximum  transmitter  moment  (from  6  A  to  12  A,  40  A  with  high-power 
option) 

•  Higher  transmitter  waveform  resolution  (from  96  kHz  to  1,536  kHz) 

•  Improved  receiver — ADC  from  48  kHz  @16  bits  to  96  kHz  @  24  bits 

•  Increased  computing  power  and  faster  data  collection — fully  30  times  a  second  up 
to  15  frequencies 

•  Advanced  display  (visual  and  audio) — a  bigger  screen 

•  Data-logging  function — other  data  including  GPS  navigation  data 

•  Increased  number  of  frequencies  for  detection  survey 

•  Integration  of  a  real-time  detection/discrimination  algorithm  (for  handheld 
operation). 

Other  EMI  technologies  that  are  suitable  for  the  UXO  problem  are  all  time-domain  systems. 
There  are  only  a  few  candidates,  the  most  common  of  which  provide  a  single  or  two  time  gates. 
The  single  time-gate  systems  provide  little  information  for  discrimination.  The  newer,  more 
advanced  time-domain  EMI  systems  provide  a  number  of  time  gates  that  in  principle  provide 
information  similar  to  a  frequency-domain  system,  but  in  order  to  match  the  bandwidth  of  the 
GEM-3,  the  decaying  transient  signal  from  eddy  currents  induced  in  a  target  must  be  observed 
over  several  decades  of  decay.  Non-EMI  technology  applicable  to  UXO  detection  consists  of 
various  forms  of  magnetometers.  Magnetic  data  provides  limited  discrimination  capability 
(some  capability  to  estimate  size,  depth,  and  possibly  aspect  ratio,  but  cannot  detect  nonferrous 
UXO)  and  is  quite  sensitive  to  geology  having  significant  magnetic  susceptibility. 

Limitations  of  this  technology  include  the  inherent  difficulty  in  distinguishing  metal  objects  by 
their  EMI  response;  objects  similar  in  size  and  shape  generate  similar  responses,  and  even 
dissimilarly  shaped  objects  may  produce  similar  responses  if  the  depth  and  orientation  is  within 
some  particular  range.  Other  limitations  include  difficulty  in  obtaining  good  s/n  while  surveying 
over  rough  terrain  and  surveying  in  heavy  brush,  dense  forest,  or  steep  terrain.  Extreme 
magnetic-soil  conditions  also  degrade  data,  though  not  to  the  point  of  inoperability.  Maintaining 
adequate  spatial  coverage  while  meeting  survey  production  needs  over  large  areas  is  also  a 
challenge. 
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3.0  DEMONSTRATION  DESIGN 


3.1  PERFORMANCE  OBJECTIVES 

The  standard  performance  metrics  for  UXO  detection/discrimination  technology  are  shown  in 
Table  1.  Since  the  operators  were  the  demonstrators  for  this  demonstration,  operator  acceptance 
may  be  interpreted  as  evaluation  by  on-site  Aberdeen  Test  Center  (ATC)  personnel,  or  their 
responsible  parties  in  charge  of  demonstration  oversight.  Such  evaluation  was  made  by 
observation  of  production  rates  and  field  problems  that  arose.  The  quantitative  objectives 
performance  was  determined  by  ATC,  based  on  the  scoring  of  the  submitted  dig  sheets. 

Table  1.  Performance  Objectives. 


Type  of 
Performance 
Objective 

Primary 

Performance 

Criteria 

Expected 

Performance 

(Metric) 

Actual 
Performance 
(Objective  met?) 

Qualitative 

Ease  of  use 

Operator  acceptance 

Acceptable 

Field  worthiness 

Operator  acceptance 

Acceptable 

Quantitative 

%  detected 

>  95% 

15%-85% 

False  alarms 

<.l 

.30-. 70 

3.2  SELECTING  TEST  SITES 

The  fonnal  demonstration  culminating  this  project  was  performed  at  the  ATC  facility  at  APG. 
Prior  to  the  demonstration  at  APG,  one  was  performed  at  a  test  grid  on  Kaho’olawe,  Hawaii,  an 
active  cleanup  site. 

3.3  TEST  SITE  HISTORY/CHARACTERISTICS 
Kaho’olawe 

The  Hawaiian  island  of  Kaho’olawe  was  used  by  the  military  for  ordnance  testing  and  as  a 
practice  range  from  World  War  II  to  1993,  at  which  time  Congress  passed  a  law  to  return  it  to  the 
state  and  to  native  Hawaiians.  PACDIV  has  been  given  the  task  of  UXO  cleanup.  To  that  end, 
clearing  UXO  has  been  ongoing,  primarily  surface  clearing  but  including  subsurface  in  select 
areas.  Since  it  is  a  volcanic  island,  the  magnetic  properties  of  the  rock  and  soil  are  mostly  high 
susceptibility  and  remnant  magnetization,  making  use  of  magnetometers  marginally  useful  for 
finding  buried  UXO.  A  seeded  test  site  has  been  created  in  a  cleared  area  to  test  methodologies 
for  buried  UXO  detection  in  the  geologic  environment  representative  of  the  island.  This  site  is 
considered  a  stringent  testing  scenario  for  UXO  detection  and  identification  in  magnetic  geology. 
Four  30m  x  30m  calibration  grids  and  twenty-two  30m  x  30m  blind  test  grids  were  used  for  this 
demonstration. 

APG 

The  APG  Standardized  UXO  Technology  Demonstration  Site  is  one  of  three  recently  completed 
facilities  designed  to  provide  UXO  detection  and  discrimination  technologies  test  scenarios  that 
evaluate  the  performance  and  operational  usability  under  the  realistic  range  of  conditions  that 
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will  be  met  during  assessment  and  clearance  operations.  These  conditions  include  various 
vegetative  states  from  barren  to  moderate  brush  to  densely  wooded  and  various  terrain  conditions 
from  open  and  flat  to  rugged.  These  conditions  provide  opportunity  for  vehicular  towed  systems, 
manual  pushcart  systems,  and  handheld  systems.  The  size  of  the  facility  is  sufficient  to  provide 
meaningful  performance  metrics  such  as  probability  of  detection,  false  alarm  rates,  and 
production  rates. 

The  choice  of  the  facility  at  Aberdeen,  in  Harford  County,  Maryland,  was  made  for  proximity  to 
the  operator’s  location  of  business  (Raleigh,  North  Carolina),  and  facility  availability. 

APG  is  an  Army  facility  that  has  been  used  for  weapons  and  military  vehicle  testing  since  1917. 
It  encompasses  117  km2  of  land,  much  of  it  forested,  between  Baltimore,  Maryland  and 
Philadelphia,  Pennsylvania.  The  UXO  demonstration  site  is  a  seeded  site  for  controlled  testing, 
and  includes  (1)  calibration  lanes  (ground  truth  revealed)  for  system  training  and  target 
characterization,  and  a  set  of  blind  (ground  truth  withheld)  areas  for  testing  a  range  of  scenarios; 
(2)  blind  test  grid — a  1,600  m2  rectangular  grid  including  access  lanes  separating  400  discrete 
lmx  lm  square  interrogation  points;  (3)  open  road  terrain — large  area  that  can  be  surveyed  with 
vehicular  towed  systems,  some  varied,  moderately  rough  terrain  and  vegetation;  (4)  moguls — an 
area  with  moguls  and  craters  of  ±lm  vertical  relief,  requiring  manual  data  acquisition,  likely 
handheld  sensor  configuration;  (5)  wooded — various  vegetation  including  significant  areas  of 
dense  trees  (Figure  6). 


Figure  6.  Aerial  Photograph  of  the  APG  Standardized  UXO  Technology  Demonstration  Site. 

Both  the  handheld  and  cart-mounted  systems  were  tested  and  “trained”  (target  libraries  collected) 
in  the  calibration  area  as  well  as  with  in-air  training  using  sample  UXO  provided  by  ATC.  The 
blind  test  grid  contains  400  actual  interrogation  squares  that  were  tested  using  the  handheld 
configuration.  The  open  field  was  surveyed  with  the  cart-mounted  system  at  0.5m  line  spacing 
with  DGPS  logged  by  the  GEM  electronics  for  georeferencing  the  data;  all  data  were  stored  in 
the  console  flash  memory  and/or  the  user  interface  for  later  processing.  The  mogul  and  wooded 
areas  were  surveyed  with  the  handheld  configuration  in  a  sweep-search  fashion  along  2m-wide 
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lanes  marked  with  cord,  with  real-time  detection  and  identification  active;  detected  targets  were 
“flagged”  for  follow-up  geo  location,  and  DGPS  data  were  collected  over  the  target. 

3.4  PHYSICAL  SETUP  AND  OPERATION 

Kaho’olawe 

Equipment  setup  included  assembly  of  the  wheeled  pushcarts  (two)  with  the  GEM-3  disks  and 
consoles  and  DGPS  rovers,  and  the  DGPS  base  station.  Some  ordnance  targets  were  provided 
for  training  on  site  (some  had  been  provided  at  Geophex  prior  to  mobilization).  The  first  day  on 
site  was  spent  in  the  calibration/test  grids,  where  ground  truth  was  provided.  Data  were  collected 
and  detected  anomalies  correlated  with  ground-truth  target  locations. 

APG 

Equipment  setup  included  assembly  of  the  wheeled  pushcart  and  ATV  sled  and  mounting  the 
GEM-3  and  related  hardware  on  the  ATV,  including  the  navigation  computer,  the  96cm  sensor 
system  with  electronics  console,  DGPS,  and  user  interface.  Similarly,  the  handheld  systems 
were  assembled,  and  DGPS  interfaced  with  an  iPAQ™  for  locating  the  flags.  The  DGPS  base 
station  was  initially  set  up  at  the  United  States  Geological  Survey  (USGS)  reference  point,  and  a 
rover  was  used  to  establish  a  more  convenient  reference  point  near  the  trailer,  for  line-of-site 
radio  communication  with  the  mobile  units.  The  systems,  including  DGPS,  were  powered  up 
and  checked  for  normal  functionality,  and  ambient  noise  levels  were  observed.  The  GEM-3 
calibration  was  verified  with  a  ferrite  rod  sample  target  for  all  systems.  Survey  line  start/stop 
points  for  the  pushcart  were  laid  out,  and  lanes  for  the  mogul  and  wooded  areas  were  marked 
with  string.  The  ATV  navigation  computer  was  set  up  with  the  site  boundaries  and  planned  line 
direction/spacing  information. 

Initially,  we  used  a  two-wheeled  trailer  cart  for  towing  with  the  ATV  but  found  this  to  be 
somewhat  unstable  over  ruts  and  gullies  and  difficult  to  turn  within  the  safety  boundaries.  A  sled 
that  had  been  made  at  Geophex  for  snow  conditions  (as  encountered  during  the  moguls  survey) 
was  delivered  to  the  site  and  replaced  the  wheeled  cart  after  confirming  that  it  could  be  pulled 
over  grass  with  better  stability  than  the  wheeled  cart. 

Handheld  sensors  used  in  the  moguls  and  woods  areas  required  no  on-site  setup. 

3.4.1  Period  of  Operation 

The  period  of  operation  at  Kaho’olawe  was  October  8-21,  2001;  no  work  was  done  on 
weekends.  The  period  of  operation  at  APG  for  the  Moguls  area  only  was  December  9-13,  2002; 
and  for  the  remaining  areas,  April  28-May  7,  2003.  A  repeat  demonstration,  funded  by  the 
ATC,  in  the  moguls  and  (reseeded)  blind  grid  with  the  handheld  GEM-3  was  performed  during 
the  periods  of  April  18-22,  2005,  and  April  25-27,  2005. 
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3.4.2  Area  Characterized 


The  test  area  at  Kaho’olawe  included  30m  x  30m  areas  designated  A-l,  A-2,  B-l  to  B-5,  through 
E-l  to  E-5,  totaling  22  sections  equal  to  1.98  hectares.  At  APG,  all  areas,  including  the  blind  test 
grid  (area  2),  open  field  (area  3),  rough  mogul  scenario  (area  4),  and  the  wooded  area  (area  5), 
were  surveyed,  for  a  total  of  approximately  6.5  hectares.  The  blind  grid  was  surveyed  three 
times  -  once  with  the  handheld  sensor  and  twice  with  the  96cm  sensor,  statically  with  the 
pushcart  and  dynamically  while  towing  with  the  ATV.  The  calibration  area  was  used  only  for 
training  and  testing,  and  data  were  not  generally  recorded;  half  of  the  calibration  area  was 
included  in  the  ATV  towed  survey  over  the  open  area  and  recorded.  The  moguls  and  (reseeded) 
blind  grid  areas  were  repeated  in  2005  with  only  minor  changes  to  the  technology,  mostly  related 
to  software. 

3.4.3  Operating  Parameters  for  the  Technology 
Kaho’olawe 

The  48cm-radius  GEM-3  sensor  disk  configuration  mounted  on  two-wheeled  pushcarts  was  used 
for  all  areas  at  Kaho’olawe.  Ten  frequencies  were  recorded  simultaneously — 90,  150,  390,  750, 
1,470,  2,970,  5,910,  1 1,910,  23,850,  and  47,970  Hz,  sampled  over  1/15  s  at  15  Hz  repetition  (i.e., 
continuous).  The  carts  were  pushed  at  typical  walking  speeds,  with  the  disk  approximately  20cm 
above  the  ground,  along  survey  lines  spaced  0.5m  apart.  Two  systems  were  operated 
simultaneously  during  most  of  the  survey,  with  a  minimum  30m  separation  to  avoid  interference. 
Each  30m  x  30m  grid  was  surveyed  between  data  downloads  (initially  two  areas  were  surveyed 
at  a  time),  with  about  10  minutes  time  required  to  download  data  and  change  batteries  between 
areas. 

APG 

Ten  frequencies  were  recorded  simultaneously — 90,  210,  390,  750,  1,470,  2,910,  5,850,  11,430, 
21,690,  and  41,010  Hz,  for  all  sensors;  the  motivation  for  the  adjustment  from  Kaho’olawe  was 
to  back  off  from  the  upper  limit  of  the  system  where  the  system  response  begins  to  roll  off  and  to 
replace  150  Hz  with  210  Hz  where  it  is  expected  to  have  less  cart-bounce  induced  noise  (90  Hz 
is  retained  for  potential  discrimination  value).  The  96cm  sensor  recorded  continuously  15  Hz 
samples  (each  integrated  over  66.7  ms),  whereas  the  handheld  operated  at  5  Hz  continuous 
sampling  (200  ms  sample  duration).  The  wheeled-cart  mounted  disk  was  about  30cm  above 
ground,  the  ATV  towed  sled  at  about  12cm,  and  the  handheld  was  operated  3 -7cm  depending  on 
snow,  vegetation,  water,  etc.  Two  handheld  sensors  were  operated  simultaneously  during  both 
the  moguls  surveys  (over  snow,  December  2002)  and  wooded  area  survey  (water-covered  terrain, 
April  2003),  with  a  minimum  sensor  separation  of  20m  to  avoid  interference.  We  operated  the 
wheeled  cart  configuration  and  the  ATV  towed  configuration  simultaneously,  with  30m 
separation  required,  and  also  while  the  wooded  area  was  being  surveyed.  Nominal  line  spacing 
was  0.5m  for  the  large-disk  surveys,  and  for  the  handheld,  sweeps  were  made  such  that  most  of 
the  area  was  covered  by  the  40cm-diameter  disk. 
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4.0  PERFORMANCE  ASSESSMENT 


4.1  PERFORMANCE  DATA 

Table  2.  Expected  Performance  and  Confirmation  Methods. 


Performance  Criteria 

Expected 

Confirmation 

Method 

Actual 

Kaho’olawe 

Response  Stage 

(1)  Probability  of  detection 

(2)  Probability  of  false  alarms 

(3)  Background  alarms 

(1)  >.90 

(2)  >  90%  of  total  metal  targets  -  UXO 

(3)  <  .025/m2 

Government 

evaluation 

(1)  .43 

(2)  .43 

(3)  547/19800m2 
=  ,0276/m2 

Discrimination  Stage 

(1)  Probability  of  detection 

(2)  Probability  of  false  alarms 

(3)  Background  alarms 

(4)  Efficiency 

(5)  Rejection  ratio 

(1)  >80% 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

(3)  <  .01/m2 

(4)  .9 

(5)  >0.5 

Government 

evaluation 

(1)  .40 

(2)  .39 

(3)  496/19800m2 
=  . 025/m2 

(4)  .94 

(5)  .07 

APG  Blind  Grid — Handheld  Configuration 

Response  Stage 

(1)  Probability  of  detection 

(2)  Probability  of  false  alarms 

(3)  Probability  of  background 
alarms 

(1)  >95% 

(2)  >  90%  of  total  metal  targets  -  UXO 

(3)  <.01 

Government 

evaluation 

(1)  .80 

(2)  .85 

(3)  .30 

Discrimination  Stage 

(1)  Probability  of  detection 

(2)  Probability  of  false  alarms 

(3)  Probability  of  background 
alarms 

(4)  Efficiency 

(5)  Rejection  ratio 

(1)  >85% 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

(3)  <.01 

(4)  0.9 

(5)  >0.5 

Government 

evaluation 

(1)  .60 

(2)  .50 

(3)  .15 

(4)  .76 

(5)  .40 

APG  Blind  Grid — 96cm-Disk,  Pushcart  Mounted  Configuration 

Response  Stage 

(1)  Probability  of  detection 

(2)  Probability  of  false  alarms 

(3)  Probability  of  background 
alarms 

(1)  >95% 

(2)  >  95%  of  total  metal  targets  -  UXO 

(3)  <.01 

Government 

evaluation 

(1)  .85 

(2)  .85 

(3)  .40 

Discrimination  Stage 

(1)  Probability  of  detection 

(2)  Probability  of  false  alarms 

(3)  Probability  of  background 
alarms 

(4)  Efficiency 

(5)  Rejection  ratio 

(1)  >85% 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

(3)  <.01 

(4)  0.9 

(5)  >0.5 

Government 

evaluation 

(1)  .70 

(2)  .70 

(3)  .35 

(4)  .82 

(5)  .19 

APG  Blind 

Grid — 96cm-Disk,  ATV  Vehicle  Towed  Configuration 

Response  Stage 

(1)  Probability  of  detection 

(2)  Probability  of  false  alarms 

(3)  Probability  of  background 
alarms 

(1)  >90% 

(2)  >  90%  of  total  metal  targets  -  UXO 

(3)  <.01 

Government 

evaluation 

(1)  .30 

(2)  .40 

(3)  0.0 
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Table  2.  Expected  Performance  and  Confirmation  Methods  ( continued ). 


Performance  Criteria 

Expected 

Confirmation 

Method 

Actual 

Discrimination  Stage 

(1)  Probability  of  detection 

(1)  >80% 

(1)  .20 

(2)  Probability  of  false  alarms 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

(2)  .30 

(3)  Probability  of  background 

(3)  <.01 

(3)  0.0 

alarms 

(4)  Efficiency 

(4)  0.9 

(4)  .71 

(5)  Rejection  ratio 

(5)  >0.5 

(5)  .22 

3 

tPG  Moguls  Area — Handheld  Configuration 

Response  Stage 

(1)  Probability  of  detection 

(1)  >90% 

Government 

(1)  -15 

(2)  Probability  of  false  alarms 

(2)  >  90%  of  total  metal  targets  -  UXO 

evaluation 

(2)  .15 

(3)  Background  alarms 

(3)  <  .01/m2 

(3)  .15 

Discrimination  Stage 

(1)  Probability  of  detection 

(1)  >80% 

(1)  .00 

(2)  Probability  of  false  alarms 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

Government 

(2)  .00 

(3)  Background  alarms 

(3)  <  .01/m2 

evaluation 

(3)  .05 

(4)  Efficiency 

(4)  0.9 

(4)  .04 

(5)  Rejection  ratio 

(5)  >0.5 

(5)  .87 

APG  Woods  Area — Handheld  Configuration 

Response  Stage 

(1)  Probability  of  detection 

(1)  >90% 

Government 

(1)  .25 

(2)  Probability  of  false  alarms 

(2)  >  90%  of  total  metal  targets  -  UXO 

evaluation 

(2)  .20 

(3)  Background  alarms 

(3)  <  .01/m2 

(3)  .05 

Discrimination  Stage 

(1)  Probability  of  detection 

(1)  >80% 

(1)  .20 

(2)  Probability  of  false  alarms 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

Government 

(2)  .10 

(3)  Background  alarms 

(3)  <  .01/m2 

evaluation 

(3)  .05 

(4)  Efficiency 

(4)  0.9 

(4)  .74 

(5)  Rejection  ratio 

(5)  >0.5 

(5)  .45 

APG  Open  Field  Area — 96cm-Disk,  Pushcart  and  ATV  Towed  Configurations 

Response  Stage 

(1)  Probability  of  detection 

(1)  >90% 

Government 

(1)  .35 

(2)  Probability  of  false  alarms 

(2)  >  90%  of  total  metal  targets  -  UXO 

evaluation 

(2)  .30 

(3)  Background  alarms 

(3)  <  .01/m2 

(3)  .05 

Discrimination  Stage 

(1)  Probability  of  detection 

(1)  >80% 

(1)  .25 

(2)  Probability  of  false  alarms 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

Government 

(2)  .15 

(3)  Background  alarms 

(3)  <  .01/m2 

evaluation 

(3)  0.0 

(4)  Efficiency 

(4)  0.9 

(4)  .68 

(5)  Rejection  ratio 

(5)  >0.5 

(5)  .44 

APG — All  Areas/Configurations,  2003 

Reliability 

No  downtime 

Government  log 

Minor  repairs 

Maintenance 

Battery  charging 

Government  log 

~4/day 

Ease  of  use 

Minimal  training 

Operator 

Minimal  training 

Factors  affecting  performance 

Some  sensitivity  to  soil  susceptibility 

Theory 

and  conductivity  and  target  depth 

Versatility 

Other  metal  detection 

Other  programs 

Underwater  UXO 
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Table  2.  Expected  Performance  and  Confirmation  Methods  ( continued ). 


Performance  Criteria 

Expected 

Confirmation 

Method 

Actual 

APG  Blind  Grid  -  Hand  Held  Configuration,  2005  repeat 

Response  Stage 

(1)  Probability  of  detection 

(1)  >95% 

(1)  .80 

(2)  Probability  of  false  alarms 

(2)  >  90%  of  total  metal  targets  -  UXO 

(2)  .85 

(3)  Probability  of  background 

(3)  <.01 

(3)  .05 

alarms 

Discrimination  Stage 

(1)  Probability  of  detection 

(1)  >85% 

(1)  .55 

(2)  Probability  of  false  alarms 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

(2)  .50 

(3)  Probability  of  background 

(3)  <.01 

(3)  .05 

alarms 

(4)  Efficiency 

(4)  0.9 

(4)  .67 

(5)  Rejection  ratio 

(5)  >0.5 

(5)  .41 

APG 

Moguls  -  Hand  Held  Configuration,  2005  repeat 

Response  Stage 

(1)  Probability  of  detection 

(1)  >95% 

Government 

(1)  -70 

(2)  Probability  of  false  alarms 

(2)  >  90%  of  total  metal  targets  -  UXO 

evaluation 

(2)  .65 

(3)  Background  alarms 

(3)  <  .01/m2 

(3)  3.0 

Discrimination  Stage 

(1)  Probability  of  detection 

(1)  >85% 

(1)  .50 

(2)  Probability  of  false  alarms 

(2)  <  .2  (<  20%  clutter  declared  UXO) 

Government 

(2)  .40 

(3)  Background  alarms 

(3)  <  .01/m2 

evaluation 

(3)  1.45 

(4)  Efficiency 

(4)  0.9 

(4)  .76 

(5)  Rejection  ratio 

(5)  >0.5 

(5)  .39 

The  tables  above  describe  performance  at  the  design  thresholds  (for  response  stage  target  and 
discrimination  stage  clutter)  chosen  by  the  demonstrator.  In  order  to  describe  the  performance 
potential  over  a  range  of  thresholds,  receiver  operator  characteristics  (ROC)  curves  generated  by 
the  government  are  reproduced  in  the  appendix  of  the  Final  Report. 

4.2  PERFORMANCE  CRITERIA 

The  performance  criteria  for  the  assessment  of  the  technology  under  demonstration  are  described 
in  Table  3. 


Table  3.  Performance  Criteria. 


Performance  Criteria 

Description 

Primary  Or  Secondary 

Probability  of  detection 

#  UXO  detected  /  #  UXO  buried 

Primary 

False  alarm  rate 

#  anomalies  not  ordnance/m2 

Reliability 

Downtime 

Maintenance 

Frequency,  required  training 

Ease  of  use 

Operator  productivity 

Factors  affecting  performance 

Operating  conditions  affecting  performance 

Versatility 

Other  potential  applications 

Secondary 
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4.3  DATA  ASSESSMENT 


During  “sweep-search”  mode,  real-time  detection  and  target  identification  were  performed,  with 
a  goodness-of-fit  (i.e.,  mean  absolute  error)  reported  for  confidence  ranking  and  clutter 
classification  (discrimination  stage).  The  response  stage  was  indicated  by  the  operator  based  on 
audio  indication  of  amplitude.  The  results  were  manually  logged  for  generation  of  dig  lists  after 
georeferencing  the  targets  with  DGPS.  Data  immediately  over  the  target,  with  background 
removed,  were  recorded  by  the  iPAQ™,  and  subsequent  reprocessing  of  the  recorded  data  was 
an  option. 

Following  postprocessing  survey  data  with  target-picking  and  discrimination  algorithms, 
prioritized  dig  sheets  were  submitted.  These  dig  lists  include  (1)  Grid  square  identification  for 
the  blind  grid  and  UTM  coordinates  for  the  other  areas,  (2)  Response  Stage  -  detection  signal 
level  (i.e.,  the  output  above  background  of  the  selected  detection  algorithm,  e.g.  quadrature  sum), 
(3)  Discrimination  Stage  Ranking  -  a  numerical  ranking  of  likelihood  that  target  UXO,  (4) 
Classification  -  Empty,  Clutter,  or  Ordnance  Type  -  identify  if  ordnance,  Depth,  Azimuth,  and 
Dip.  The  last  two  items  may  not  be  an  output  of  the  discrimination  algorithm  used. 

4.3.1  Performance 

Kaho’olawe 

A  detailed  self-evaluation  of  the  Kaho’olawe  demonstration  performance  was  submitted  to  the 
ESTCP  in  January  2003;  a  summary  of  key  points  follows. 

Of  148  ordnance  items  not  designated  non-grade,  91  were  detected  and  declared  ordnance  with 
varying  confidence,  but  26  of  those  were  outside  the  0.5m  allowed  position  error  radius  (but 
within  1.0m);  there  was  no  data  (i.e.,  an  obstacle  blocking  the  cart  or  surveying  error)  over  seven 
items.  Of  the  50  ordnance  items  completely  missed  and  data  were  available,  14  were  20mm  and 
1 1  were  40mm  items  (only  3  of  the  20mm  were  detected,  and  7  of  the  40mm),  indicating  that  the 
small  items  were  problematic  at  Kaho’olawe. 

Of  ordnance  declared  non-grade  (not  defined  in  ground-truth  tables  but  assumed  to  mean 
ordnance  with  missing  parts  but  identifiable),  21  were  detected  and  four  missed.  None  of  the 
ordnance  (including  non-grade)  was  classified  as  clutter.  Of  158  seeded  fragments,  67  were 
detected,  five  of  which  were  classified  as  clutter.  At  Kaho’olawe,  our  ordnance/clutter  threshold 
was  excessively  conservative  and  we  thus  classified  most  metal  objects  as  ordnance. 

There  were  a  large  number  of  picked  targets  (547)  that  did  not  correspond  to  seeded  or  known 
metal  objects,  and  of  these  496  were  classified  as  ordnance,  resulting  in  a  very  large  false 
background  score.  However,  there  were  a  number  of  instances  where  multiple  demonstrators 
identified  the  same  target,  and  when  followed  up  with  a  ground  search,  a  metal  object  was 
usually  found.  Ground  follow-up  was  undertaken  only  if  at  least  four  demonstrators  had 
coinciding  targets.  We  were  convinced  that  a  large  fraction  of  our  false  backgrounds  were 
caused  by  metal  objects  because  many  were  associated  with  high  s/n  spectra  clearly 
characteristic  of  metal  (strong  quadrature  peak  below  10  kHz). 
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Although  Kaho’olawe  poses  a  difficult  geologic  environment,  the  extreme  magnetic  geology  was 
not  the  most  important  factor  in  our  results.  There  was  no  correlation  between  the  level  of 
background  susceptibility  (which  varied  significantly)  and  our  performance,  and  very  few 
features  in  the  processed  detection  channel  obscuring  anomalies  that  could  be  correlated  to 
magnetic  susceptibility. 

On  the  other  hand,  we  found  that  the  low-frequency  data  (90  Hz  and  150  Hz)  that  are  particularly 
valuable  for  metallic  targets  were  corrupted  by  platform  motion,  which  we  have  since  identified 
as  angular  motion  of  the  receiver  coil  in  the  earth’s  magnetic  field  as  the  cart  is  rolled  over  rough 
ground.  We  also  had  navigation  errors,  both  in  terms  of  locating  the  target  within  the 
0.5-m-halo,  and  in  terms  of  the  operator  maintaining  the  0.5m-line  spacing.  The  latter  impacts 
our  ability  to  find  small  targets  (20mm  and  40mm)  because  line  gaps  ~0.8m  were  not 
uncommon;  also,  positioning  targets  from  anomalies  centered  within  these  gaps  is  subject  to 
greater  error. 

APG 

The  performance  shown  in  the  tables  did  not  meet  expectations,  particularly  for  the  PD  with  the 
handheld  system  in  the  moguls  and  woods.  We  believe  that  a  systematic  problem  with  locating 
the  target  flags  resulted  in  reported  position  coordinates  outside  the  0.5m  allowed  halo;  verbal 
feedback  from  ATC  indicated  that  a  2-  to  3-foot  shift  in  all  target  positions  in  the  mogul  area 
increased  the  PD  to  about  0.4,  still  below  expectations  but  more  reasonable.  A  problem  with  the 
differential  corrections  may  have  occurred  and  may  not  have  been  exactly  the  same  over  the 
duration  of  the  survey  (so  that  one  simple  shift  of  all  targets  would  not  achieve  the  true  sensor 
PD).  These  are  speculative  explanations,  and  only  repeat  experiments  can  ascertain  the  GEM-3 
performance.  Preliminary  results  from  a  repeat  survey  performed  in  2005  over  the  mogul  area 
achieved  a  .7  response  stage  PD. 

The  performance  of  the  ATV  towed  system  also  was  well  below  expectations.  This  may  be 
attributed  to  a  combination  of  positioning  errors  and  data  quality  issues  associated  with  a  sensor 
towed  at  ~5  mph  along  unmarked  traverses  (some  gaps  in  coverage  were  evident  on  the  anomaly 
maps).  The  nominal  0.5m  line  spacing  is  difficult  to  maintain,  and  improvements  in  operator 
queued  navigation  is  needed.  Motion-induced  noise  is  also  greater  in  the  towed  configuration. 

Discrimination,  based  on  a  simple  single-point  (spatial)  matching  algorithm,  has  not  proved  to 
reduce  FAR  while  maintaining  PD  except  at  very  low  thresholds  with  associated  low  PD.  The 
best  discrimination  performance  occurred  with  handheld  and  pushcart  configurations  in  the  blind 
grid,  where  data  were  collected  statically  over  the  target.  Survey  (ATV-towed)  mode  degrades 
the  discrimination  performance,  and  the  handheld  in  moguls  and  woods  had  a  systemic  problem 
with  target  detection/location  that  rendered  discrimination  perfonnance  invalid. 

4.4  TECHNOLOGY  COMPARISON 

No  performance  data  for  other  technology  demonstrators  are  available. 
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5.0 


COST  ASSESSMENT 


5.1  COST  REPORTING 

Cost  reporting  consists  of  tracking  production  rates  for  each  scenario,  mobilization,  and  setup 
hours,  including  hours  of  training  over  calibration  lanes.  An  itemized  cost  estimate  for  the 
demonstration,  produced  by  ATC  based  on  observer  logs,  is  provided  in  Table  4.  Costs  for  the 
various  scenarios  of  the  demonstration  were  combined. 

Table  4.  Demonstration  Costs. 


Equipment  Costs  (rental  -  variable) 

Item 

No. 

Rate/Week  ($) 

Weeks 

Cost  ($) 

Handheld  GEM 

2 

500 

2 

2,000.00 

Towed  GEM 

1 

750 

1.5 

1,075.00 

DGPS 

2 

500 

2 

2,000.00 

ATV 

1 

300 

1.5 

450.00 

Subtotal 

$5,525.00 

No.  People 

Hourly  Wage 

Hours 

Cost 

Initial  Setup  (Fixed) 

Supervisor 

1 

95.00 

8.5 

807.50 

Data  analyst 

1 

57.00 

8.5 

484.50 

Field  support 

1 

28.50 

8.5 

242.25 

Subtotal 

$1,534.25 

Calibration  (Variable) 

Supervisor 

1 

95.00 

10.1 

959.50 

Data  analyst 

1 

57.00 

10.1 

575.70 

Field  support 

1 

28.50 

10.1 

287.85 

Subtotal 

$1,823.05 

Site  Survey  (Variable) 

Supervisor 

1 

95.00 

100 

9,500.00 

Data  analyst 

1 

57.00 

100 

5,700.00 

Field  support 

5 

28.50 

100 

14,250.00 

Subtotal 

$29,450.00 

Demobilization  (Fixed) 

Supervisor 

1 

95.00 

1.28 

121.60 

Data  analyst 

1 

57.00 

1.28 

72.96 

Field  support 

1 

28.50 

1.28 

36.48 

Subtotal 

$231.04 

Total 

$38,563.34 
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5.2  COST  ANALYSIS 


The  demonstration  costs  shown  in  Table  4  are  used  to  estimate  production  costs  for  an 
operational  mission.  Anticipated  costs  of  GEM-3  system  and  support  equipment  for  purchase 
and  or  rental  will  be  estimated. 

Initial  setup,  calibration,  demobilization,  and  equipment  are  based  on  the  premise  that  both  hand¬ 
held  and  towed  systems  are  used,  i.e.,  the  mission  includes  mixed  conditions  requiring  use  of 
both  configurations. 


Table  5.  Operational  Costs  Estimate. 


Initial  setup 

Mobilization 

$1,500 

GEM-3  target  training 

$1,800 

Site  survey 

Data  acquisition 

$1, 500/hectare 

Data  analysis 

$5  00/hectare 

Demobilization 

$800 

Equipment  rental 

4  handheld  +  towed  GEMs 

$4,000/wk 

5.3  COST  COMPARISON 

Costs  for  conventional  technologies  for  this  site  are  not  known.  It  is  likely  that  in  practice  costs 
of  conventional  technologies  would  be  similar — mag  and  flag  less  in  the  moguls  and  woods  but 
more  in  the  open  area.  The  handheld  GEM-3  operates  similarly  to  any  conventional  handheld 
technology  during  the  search-and-detect  time,  but  requires  2  seconds  of  static  data  collection  to 
acquire  discrimination  spectra,  and  sometimes  requires  a  background  reading  as  well.  The  cart- 
mounted  GEM-3  operates  similarly  to  an  EM-61,  requiring  a  half-meter  line  spacing  and 
surveying  at  a  normal  walking  speed;  the  ATV  towed  version  is  similar.  The  added  cost  of 
discrimination  processing  is  small  (done  in  real-time  for  the  handheld  and  automated  batch  mode 
for  the  cart/sled  configuration). 

The  savings  is  realized  when  false  alarms  have  been  reduced  resulting  in  fewer  unnecessary 
follow-up  digs,  typically  costing  $200/target.  As  yet,  the  reduced  false  alarms  have  not  been 
demonstrated  to  a  level  that  would  eliminate  a  significant  number  of  unnecessary  digs. 
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6.0  IMPLEMENTATION  ISSUES 


6.1  COST  OBSERVATIONS 

Some  problems  with  the  equipment  and  logistics  resulted  in  downtime  and  inefficiency  that 
would  be  reduced  with  experience,  since  this  was  the  first  use  of  all  the  equipment  except  the 
wheeled-cart  configuration.  Refinement  of  field  techniques  and  equipment  will  result  in  greater 
productivity.  Some  cost-driving  issues  were  site  specific — the  moguls  and  woods  areas  have  a 
high  density  of  targets  (more  than  800  in  the  moguls,  more  than  400  in  the  woods)  and  are 
difficult  to  survey  because  of  the  respective  “challenge”  scenario  characteristics  of  those  sites. 
In  addition  to  the  rugged  terrain  of  the  moguls,  we  perfonned  the  survey  during  winter  when 
they  were  covered  with  snow,  ice,  and  slushy  water.  The  woods  and  large  areas  of  the  open  area 
were  water  covered,  sometimes  more  than  6  inches  deep.  The  open  area  buffer  zone  was  narrow 
and  we  were  restricted  from  going  outside  that  area,  which  posed  some  survey  maneuvering 
difficulties.  We  found  that  the  large  volumes  of  data  from  the  open  area  were  difficult  to  plot  for 
on-site  quality  assurance/quality  control  (QA/QC).  Some  of  these  are  simply  challenges  posed 
by  the  site,  but  some  can  be  better  handled  after  gaining  experience. 

6.2  PERFORMANCE  OBSERVATIONS 

The  most  glaring  perfonnance  observation  was  the  poor  correlation  between  flagged  targets  in 
the  moguls  and  locations  of  seeded  targets,  and,  to  a  lesser  extent,  in  the  woods.  We  believe  we 
had  a  problem  surveying  the  DGPS  positions  of  the  flags,  possibly  with  a  base  station  error.  We 
checked  grid  comers  with  provided  coordinates,  but  evidently  it  did  not  show  the  problem  (note 
that  the  mogul  area  was  done  at  a  different  time  than  the  other  areas).  The  problem  in  the  woods 
may  have  simply  been  normal  degradation  of  DGPS  with  satellite  signal  interference  from  trees. 
(DGPS  is  still  functional,  but  conversation  with  others  who  have  worked  there  gave  reports  of 
only  ~1  m  accuracy.)  Some  navigation  issues  with  the  ATV  configuration  resulted  in  a  small 
percentage  of  the  area  missed — improvement  in  this  regard  should  be  achievable. 

The  most  apparent  deficiency  with  the  GEM  sensors  in  the  context  of  this  site  is  the  inability  to 
detect  many  of  the  deep  targets;  this  was  observed  in  the  calibration  grid  where  ground  truth  was 
known.  The  depth  limitation  scales  with  the  target  size,  so  the  medium  and  large  targets  buried 
the  greatest  gave  no  discemable  response.  We  feel  these  depths  are  likely  not  realistic  or 
representative  of  typical  depth  distributions,  but  it  would  be  useful  if  some  statistical  data  from 
actual  cleared  sites  were  readily  available  to  clarify  what  depth  objective  we  should  anticipate. 

Finally,  the  target  density  in  the  moguls  and  woods  is  extreme — discrimination  of  individual 
objects  is  probably  not  practical,  and  a  real  site  would  need  to  be  completely  excavated  down  to 
a  foot  or  so  and  then  surveyed  for  remaining  residual  items. 

6.3  SCALE-UP 

The  demonstration  scale  was  large  enough  to  provide  a  reasonable  estimate  for  full-scale 
applications.  Some  increased  efficiency  will  be  expected  with  regard  to  startup  and 
demobilization  costs,  as  well  as  improved  field  procedures  attained  from  learning  a  site.  Some 
increased  costs  could  result  for  more  remote  areas  from  logistical  costs. 
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6.4  OTHER  SIGNIFICANT  OBSERVATIONS 


Some  usability  issues  were  observed,  such  as  a  need  for  more  field-worthy  iPAQ™  connectors 
(ruggedized  versions  exist  at  greater  cost),  and  there  is  room  for  improvements  in  the  sensor 
platforms.  (The  wheeled  cart  and  the  ATV  towed  sled  which  did  not  last  the  survey  due  to 
abrasion.)  These  are  refinements  that  will  be  driven  by  user  demand  and/or  continued  Geophex 
use. 


6.5  LESSONS  LEARNED 

We  need  to  ensure  georeferencing  accuracy  with  more  rigorous  verification.  We  will  improve 
our  towed  platforms  and  ATV  navigation  aids. 

6.6  END-USER  ISSUES 

The  end  user  must  understand  the  depth  of  detection  limit  for  the  potential  UXO  at  a  given  site 
and  the  detection  depth  requirements  for  that  site  (e.g.,  a  155mm  target  cannot  be  detected  at  2m 
depth).  Discrimination  stage  performance  depends  on  the  suite  of  potential  UXO,  clutter 
characteristics,  and  target  depths;  we  have  been  significantly  more  successful  during  laboratory 
testing  with  a  more  limited  set  of  UXO  and  clutter  items  and  depths  limited  to  provide  high  s/n 
than  we  have  been  at  APG. 

6.7  APPROACH  TO  REGULATORY  COMPLIANCE  AND  ACCEPTANCE 

The  only  regulatory  compliance  issues  relate  to  site  access  and  DGPS  base  station  radio 
frequencies  used.  Acceptance  may  require  a  definitive  performance  demonstration  at  a  level  that 
achieves  the  stated  goals.  The  regulatory  acceptance  issues  for  the  technology  pertain  to  the 
intended  depth  to  be  cleared  of  UXO  and  the  depth  to  which  UXO  can  be  detected,  and  whether 
a  significant  portion  of  clutter  can  be  reliably  classified  as  such  without  reduction  in  UXO  PD, 
which  must  be  demonstrated  at  each  site  with  a  statistical  sample  of  verification  digs. 
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